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OPTIMAL  CODING  OF  OBJECTS  WHEN  CLASSIFYING  THEM  BY 
METHODS  OF  PATTERN  RECOGNITION 

Yu.  V.  Devingtal’ 

(Perm’) 

Proposed  is  a  method  that  in  some  sense  is  the  optimal  coding  of  objects  in  the 
classification  of  complex  systems  with  the  help  of  a  dividing  plane.  The  determination  of 
the  codes  is  reduced  to  the  solution  of  a  problem  of  linear  programming.  The  identification 
of  objects  not  included  in  the  teaching  sequence  consists  in  the  summation  of  codes.  The 
obtained  table  of  codes  can  be  used,  for  example,  for  determining  the  oil  content  of  beds 
from  the  data  of  geophysical  measurements. 

1.  Preliminary  observations.  The  methods  of  pattern  recognition  now  are  widely 
used  not  only  for  the  identification  of  visual  forms,  but  also  for  the  classification  of  complex 
systems  of  different  types  which  are  described  by  a  large  quantity  of  parameters  or  criteria. 
Criteria  are  either  results  of  some  quantitative  measurements  or  qualitative  evaluations.  It 
is  possible  to  assume  that  each  parameter  has  a  finite  number  of  gradations.  For  the 
continuously  varying  parameters  their  range  of  change  is  divided  into  a  finite  number  of 
intervals,  in  each  of  which  the  value  of  the  parameter  can  be  considered  constant. 


To  input  information  about  the  object  into  a  computer  each  value  of  parameter  in 
coded  by  some  method. 

If  objects  are  characterized  by  n  parameters,  each  of  which  can  take  lq  (i=l,  2, ..., 
n)  values,  then  for  coding  an  object  a  table  of  codes  preliminarily  is  compiled 

{xj  i  1*  2, ...,  nj  j — 1,  2, ...,  k;, 

where  Xy  -  number  by  which  the  jth  value  of  a  parameter  with  the  number  i  is  coded.  In 
practice  the  codes  Xy  sometimes  coincide  with  the  numerical  values  of  the  measurements  of 
parameters,  sometimes  they  are  some  conditional  numbers.  It  is  almost  always  thought  that 


l 


different  values  of  codes  must  correspond  to  the  different  values  of  parameter.  The 
selection  of  the  table  {Xy}  depends  on  the  conditions  of  a  specific  problem  and  in  many 
respects  is  determined  by  convenience  in  writing  the  codes  into  the  computer.  In  studies 
dedicated  to  pattern  recognition  it  is  usually  assumed  that  this  coding  has  already  been 
carried  out,  and  the  objects  are  considered  as  points  whose  coordinates  coincide  with  the 
values  of  the  codes.  This  space  is  called  the  space  of  criteria. 

The  simplest  method  of  classification  is  the  division  of  objects  into  two  classes  A  and 
B  with  the  help  of  a  dividing  hyperplane.  The  different  methods  of  constructing  such  a 
hyperplane  are  described,  for  example,  in  [w].  However,  not  always  is  coding  of  initial 
information  successful.  Sometimes  the  obtained  point  sets  do  not  share  one  hyperplane, 
which  strongly  narrows  the  applicability  of  this  simple  method  of  division. 
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Example.  Let  the  objects  be  characterized  by  two  parameters,  each  of  which  can 
take  three  different  values,  and  let  the  table  of  the  codes  take  the  form: 


Let  us  assign  the  teaching  sequence  this  form:  let  points  (x12,  x21),  (x12,  x^)  belong 
to  class  A,  and  points  (xu,  x^,  (x13,  x^)  -  to  class  B. 

With  this  coding  of  the  values  of  the  parameters  these  two  classes,  obviously,  cannot 
be  divided  by  one  plane.  A  similar  picture  is  also  encountered  in  practical  problems: 
sometimes  an  unsuccessful  method  of  coding  does  not  permit  the  use  of  the  dividing  plane 
for  the  classification.  Therefore,  we  naturally  have  the  problem  of  constructing  a  method 
of  coding  objects  such  that  the  teaching  sequence  in  a  sense  in  the  best  way  would  be 
divided  by  a  given  hyperplane.  In  this  study  one  method  constructing  this  table  of  codes 
{Xjj}  is  examined.  In  contrast  to  [M],  where  the  points  of  the  teaching  sequence  are 
considered  to  be  predetermined),  i.e,,  the  table  of  codes  {Xy}  has  been  selected  previously 
and  the  dividing  hyperplane  is  sought,  we  consider  the  hyperplane  the  given,  and  we  will 
seek  the  elements  of  the  table  of  codes  {xy}.  As  will  be  clear  below,  this  method  widens 
the  class  of  sets  divided  by  one  hyperplane. 
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2.  Formulation  of  the  problem.  Without  limiting  generality,  it  is  possible  to  assume 
that  the  unknown  values  of  codes  x-  satisfy  the  inequalities 

0<Xy^l  i=l,  2, n;  j=l,  2, lq.  (2.1) 

As  the  dividing  plane  let  us  take  the  plane 

r*  +  +  •  •  •  4- —  —  =  0,  (2.2) 

which  divides  in  half  an  n-dimensional  hyper  cube. 

Since  each  specific  object  is  characterized  by  certain  values  of  the  parameters,  then 
it  suffices  to  indicate,  when  specifying  an  object,  what  value  of  each  parameter  enters  into 
the  description  of  this  object,  i.e.,  it  suffices  to  assign  the  sequence  of  second  indices  j,  (i=l, 
2,  ...,  n)  in  codes  Xy  of  the  criteria  which  define  the  objects  of  the  teaching  sequence.  In 
other  words,  it  is  necessary  to  indicate  that  for  a  given  object  the  first  parameter  in  its 
description  a  takes  value  with  a  number  jt,  the  second  -  with  a  number  j2  and  so  forth. 

Let  the  teaching  sequence  be  specified  by  the  method  described  above.  Each  object 
is  written  in  the  form  of  a  set  n  of  whole  numbers  j*.  (i=l,  2, ....  n),  where  r  -  number  of  the 
object.  Let  r=l,  2,  ...,  m,  correspond  to  the  objects  of  class  A,  and  r=mi+,  mj+2,  ..., 
mj+m2  -  to  objects  of  class  B.  Let  us  denote  by  =  2  k*  the  totaI  number  of  all 

i**i 

possible  gradations  of  the  parameters;  by  eyVn  -  distance  from  dividing  plane  (2.2)  to  the 
point  of  the  teaching  sequence  nearest  to  it.  It  is  possible  to  require,  for  example,  that  the 
plane  (2.2)  divide  the  teaching  sequence  so  that  the  distance  from  the  nearest  point  of  the 
teaching  sequence  to  the  dividing  plane  would  be  greatest.  In  this  case  we  will  arrive  at  the 
following  of  linear  programming:  to  find  values  e  and  Xy  (1=1,  2, ...,  n;  j=l,  2, ...,  k,)  such 
that  e  takes  the  greatest  value  with  satisfaction  of  the  conditions 

x<#ir+ai/j,’+*"+*n/iir— e-y- 5s  0  (2.3) 

for  r=l,  2, ...,  mj; 

-  *u  sr”  Xu  tr~  ■  ■  •  —  *  +  Y  >  0  (2-3/) 


3 


for  r=m1+l,  m,+2, ...,  mj+m^  where  c^O  and  0<Xy£l. 
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Conditions  (23)  and  (23’)  mean  that  the  points  of  class  A  are  located  on  one  side  of 
the  plane  (23),  and  those  of  class  B  on  other  side  of  this  plane.  Conditions  (23)  and  (23’) 
can  be  written  in  the  form  of  one  system  of  inequalities 


where 


n 

2  2  xu  4*  e  ^  6r  — ,  r  a  2, . . . ,  -j-  mt, 

i-i  l-~i 

5r=r{-1’  r==1*2”'- 

A  1,  r=m,  +  1, 


(2.4) 


C$  = 


—  1,  r  s=s  i42, .. . ,  mu 

mt  "l-  2, . . . ,  -f-  mz, 

t .  7  ==  fir,  t  —  1,2,...,  m{ , 

—  1,7  —  fir,  r  —  mt  +  1,  m4  -f  2, . . . ,  m,  +  m2, 
0,  j  ¥=  jir ,  r  =  1,2 , . . . ,  mt  +  m:. 


3.  Solubility  of  the  problem  and  some  observations  about  its  solution.  The  problem 
of  linear  programming  formulated  above  is  always  soluble,  since  its  solution  set  is  not  empty, 
and  the  linear  form  is  limited  [5,  page  110,  theorem  4.4]. 

For  example. 


e=0,  x 1=05,  i-1, 2, n;  j=l,  2, ...,  (3.1) 

is  the  solution  of  this  problem,  and  the  linear  form  of  e,  with  the  limits  used,  satisfies  the 
inequality 

2 

The  solubility  of  a  linear  programming  problem  is  not  equivalent  to  solving  the 
problem  of  dividing  objects  into  two  classes  with  the  help  of  a  hyperplane  (2.2).  If  solution 
of  the  problem  of  linear  programming  will  give  max  e-0,  then  the  of  construction  a 
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hyperplane  which  divides  the  teaching  sequence  is  impossible  for  any  choice  of  table  {x^}. 

If  for  a  certain  coding  of  objects  {x^*}  it  is  possible  to  construct  the  dividing 
hyperplane 


apci'+aic2' + ... + a^-aQ = 0,  (3.2) 

then  there  is  always  a  coding  {x,,*}  for  which  the  teaching  sequence  is  divided  by  a 
hyperplane  (22)  with  satisfaction  of  the  conditions  (2.1). 

Actually,  since  (3.2)  is  the  dividing  plane,  then  the  coordinates  of  the  objects  of  the 
teaching  sequence  satisfy  the  inequalities 

n 

'2  aiX<)  r  —  ^  >  o 

i—t 

with  r=l,  2, ....  m( 

n 

and  2  «**,-*•  <0 

i— 1 

with  r=m,+l,  nij+2, ...,  mj+n^. 

Let  us  denote  the  largest  value  of  the  code  of  values  of  the  parameter  with  the 
number  i  in  the  training  sequence  by  =  max*Jr,  and  the  smallest  value  by 

—  x'min  =  minx,*r,  r  =  1, 2, . . . ,  mx  -f  mz. 

r 
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Without  limiting  generality,  it  is  possible  to  consider  all  a^O  i=l,  2,  n,  since,  if 
some  a*  proves  to  be  equal  to  zero,  then  the  parameter  with  number  k  does  not  participate 
in  recognition,  and  the  object  can  be  described  by  a  smaller  number  of  parameters  without 
sacrificing  the  division.  With  some  a,  <0  we  can  preliminarily  change  the  directions  of  an 
axis  having  the  number  (,  i.e.,  replace  x,’  by  -xf‘,  in  other  words,  to  change  the  sign  in  the 
codes  of  the  parameter  having  the  number  t. 


(3.3) 

(3.3') 
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In  (32)  let  us  replace  the  variables 


xi*=cpci+dii 


(S.h) 


where 


d  ^  min  ’  i 


f  inf  (a*x/min) !  —  ao  — sup(aiiimax)  ^  5  (3-5) 


i  =s  1,2, 


Substituting  (3.4)  in  (3.2),  we  will  obtain 

W  2  ** 

y,akxk’  —  a o== — (ciQ  —  nd)  y,xk+nd  —  a0=  0 
n 

or  after  division  by  2(ao-nd)/n?£0  we  will  obtain  the  equation  of  a  hyperplane  with  the  form 

(22) 

k— 1  £ 


Let  us  note  that  ao-nd>0  because  of  the  selection  of  d.  Actually,  if  the  smallest 

•  9 

value  in  (3.5)  is  inf  (a,*,  mm),  then  d  <  inf  (a.-Xi  mm)  and 

i  < 

n 

ao  —  nd  ^  Oo  —  n  inf  (a*!,- mm)  ^  —  2  mm  > 

‘  <— i 

n 

>  Oa—  2 a<xV  r>  0  (»*i  +  1  <  r  <  m,  -)-  m*) 

1 


because  of  the  relationship  (33’)- 
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In  the  second  case 


ao~  nd^  a*—  2a«  +  n  sup  ^ 

< 

»  n. 

^  —  Oo  ■+•  2  a<x*  5s  ~  <*o  +  2  a<x>i,  >0  1  ^  r  <C  m, 

<«•!  i—l 


because  of  the  relation  (33). 

Satisfaction  of  the  condition  (2.1)  also  easily  is  checked  from  the  relation  (3.4) 

(aix'itr—d)n 
X{i'  2  (a*  —  nd)  ’ 

Because  of  the  selection  of  d  xijr^>  o,  since  a<j-nd >0  and  a,*,’  —d^t  inf fa<x, mm)— 
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2aQ  —  nd—naix\  2a0  —  nd—  nSup 

Similarly,  since  l  ~  xUr  » - — hr  >  « 


2  ( a0  —  nd) 


2  (a0  —  nd) 

The  conditions  (33)  and  (33’)  with  this  replacement  will  become  the  inequalities 

Sx<;r - £>0 


>0. 


with  r=l,  2, ...»  m1?  while 


n 

2  xa 

i—i 


0 


with  r=m,+l,  m,+2, ...» m,+m2,  i.e.,  there  is  an  e>0,  such  that  the  conditions  (2.1),  (23), 
(23’)  are  satisfied,  and  therefore  the  problem  in  p.  2  will  in  this  case  be  solution  such  that 
max  e>0. 

A  change  in  the  coding  of  objects  can  convert  nondividing  sets  into  dividing  ones. 

Let  us  return  to  the  example.  In  this  case  the  problem  of  linear  progiamming  will 
consist  in  defining  seven  unknowns  e,  xn>  x12,  xJ3,  x2),  x22,  x23  so  as  to  obtain  max  e  during  the 


7 


limits 


xa  +  Xu  —  e  —  1  ^  0, 

xn  4-  *23  —  e  —  1  ^0, 

—  xji  —  ^22  —  e  4- 1  ^  0, 

—  xtj  —  X22  —  e  4- 1  >  0, 

0<xu<l,  *  =  1,2;  /  =  1,2, 3. 

It  is  not  difficult  to  verify  that  the  solution  to  this  problem  will  be  the  numbers 

0  —  1,  Xji  “ —  0,  ' —  1,  ——  0, 

X21  — '  1,  X22  0,  X23  =  1, 

and  the  plane  x,  4-x2-l  =0  will  divide  the  caching  sequence  indicated. 

From  the  example  it  is  evident  that  the  codes  of  the  different  values  of  the 
parameters  can  coincide  (xu  =x13,  x21  ^x^).  If  some  gradation  of  a  parameter  is  encountered 
only  in  class  A  (or  class  B),  then  the  corresponding  value  of  the  code  will  be  equal  to  1  (or 
respectively  to  0).  If  a  certain  gradation  is  not  encountered  in  any  of  the  classes  (this 
sometimes  happens  in  practical  problems  with  insufficient  representativeness  of  the  training 
sequence),  then  let  us  assume  the  value  of  the  corresponding  code  to  be  equal  to  0.5.  It 
should  be  stressed  that  the  solubility  of  the  of  linear  programming  problem  does  not  mean 
that  any  two  sets  can  be  divided  by  one  plane.  For  example,  if  one  and  the  same  object  will 
be  written  in  the  training  sequence  and  in  class  A  and  class  B,  then  this  will  lead  to  max 
e=0. 

Let  us  consider  conditions  under  which  the  solution  (3.1)  will  be  optimal,  i.e.,  let  us 
find  the  criterion  of  insolubility  of  the  division  problem.  For  this  let  us  switch  over  to  a  dual 
Problem.  In  it  a  minimum  with  the  linear  form 

ni| 

w  —  —  2  +  2  u*  (3-6) 
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is  sought  under  the  conditions 


m.+m, 

2 


(2-7) 


m  i+fftj 

2  Cij  ar  +  um,+m,+k,+kl+...+h._l  +]  ^  0,  (3.8) 

r— 1 

/  =  1,  2, . . . ,  ki;  i  —  1,  2, .  . ,  n,  k0  =  0 
Uk  ^  0,  k  =  t,  2, . . . ,  mi  +  +  (V. 
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According  to  the  second  duality  theorem,  if  the  solution  (3.1)  is  an  optimal  solution, 
then  the  components  uk*  of  the  solution  of  the  dual  problem  with  k>m,+m2  will  be  equal 
to  zero  (since  are  satisfied  the  conditions  (2.1)  with  absolute  inequality),  all  limimions  (3.8) 
are  converted  into  equalities  (since  x^O),  the  linear  form  (3.6)  becomes  zero  (since  6=0), 
and  the  inequality  (3.7)  can  be  written  in  the  form  of  the  equality 

2  uh  —  1 4-  \  where  6^0. 

Ammi 


Thus,  if  the  solution  (3.1)  is  optimal,  then  the  system 


m,+m, 

2  —  1  4“ 

k— 1 

-2  “*‘4-  2  —  o, 

m»+mt 

2d?ur*  =  0,  i==  1,2,  ...,n,  /=l,2,...,Jti  (3.9) 

r»»i 


has  a  non-negative  solution. 
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Let  us  denote  by  R ,  R2  and  Ru  respectively,  the  ranks  of  a  matrix  M  of  the  system 
(3.9),  its  augmented  matrix  and  matrix  M„  which  is  obtained  from  matrix  M  by  discarding 
first  line. 

Matrix  M  of  the  system  (3.9)  has  the  following  properties: 

1)  its  first  line  consists  of  m,+n2  ones; 

2)  the  second  line  has  mi  elements  equal  to  -  1,  and  m2  elements  equal  to  1; 

3)  the  remaining  N  rows  of  matrix  M  can  be  decomposed  into  n  layers;  a  layer  with 
number  i  contains  kj  lines;  i=l,  2, ...,  n.  In  each  column  of  a  layer  is  located  one 
and  only  one  nonzero  element  equal  to  - 1,  if  the  column  is  located  in  the  first  m, 
columns,  or  to  +1  for  the  remaining  columns. 

Matrix  M  coincides  with  the  transposed  augmented  matrix  of  limits  (2.4).  Let  us  note 
that  matrix  M  has  no  more  than  N+ 2-n  linearly  independent  lines,  since  the  sum  of  the 
elements  of  the  columns  of  each  zone  is  equal  to  the  second  row  of  matrix  M.  Therefore 
R<min  (N+2-n;  nq+mz).  The  rank  of  the  augmented  matrix  R2=Rl  +  l,  since  the  free 
terms  have  only  one  element  different  from  zero. 

Let  us  call  the  teaching  sequence  short  if  m1+m2<N+2-n  and  long  if  otherwise. 

From  all  of  the  aforesaid  the  following  can  be  stated. 

The  problem  o  dividing  objects  into  two  classes  with  one  hyperplane  will  be  solved 
if  one  of  two  conditions  has  been  fulfilled: 

a)  the  system  (3.9)  is  insoluble,  i.e.,  R2^R; 

b)  R2-R ,  but  any  solution  of  system  (3.9)  has  at  least  one  negative  component. 

For  the  short  teaching  sequence  the  problem  of  division,  in  particular,  will  be  soluble 
if  i?,=m1+m2,  since  in  this  case  R=RU  while  R2=RX+ 1,  and  system  (3.9)  will  be  inconsistent. 
When  /?,<mi+m2  and  /?=/?, +1,  the  problem  of  division  is  soluble  if  there  are  different 
signs  for  the  cofactors  of  a  line  consisting  entirely  of  ones  of  a  nonzero  determinant  of  order 
R  (in  this  case  the  solution  of  system  (3.9)  will  have  negative  components). 
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The  problem  of  linear  programming  examined  is  characterized  by  great 
dimensionality  (N+ 1  unknowns,  m,+m2 +N  limits)  and  by  the  comparatively  simple  structure 
of  the  matrix  of  limits. 

Each  row  of  the  matrix  of  limits  (23)  and  (23’)  containing  n  nonzero  elements  can 
be  written  in  computer  memory  in  the  form  of  numbers  of  columns  j„  in  which  are  these 
elements  are  located 

For  solution  of  this  problem  it  is  advisable  to  use  methods  which  do  not  revise  the 
initial  matrix  of  limits.  This  condition  is  satisfied,  for  example,  by  the  method  of 
Pietszykowski  [*].  As  the  initial  approximation  it  is  useful  to  adopt  the  solution  (3.1). 

Those  gradations  of  parameters  which  did  not  belong  to  the  teaching  sequence  will 
preserve  in  the  solution  values  equal  to  03.  Since  in  the  teaching  sequence  there  is  no 
information  about  the  fact  that  such  gradations  of  parameters  are  encountered  more 
frequently  in  one  of  the  classes  than  in  other,  then  there  are  no  bases  for  coding  them  by 
other  numbers. 

The  value  of  the  code  will  be  more  than  0.5  when  this  gradation  of  the  parameter 
more  frequently  is  encountered  in  class  A  and  less  than  0.5  if  it  is  characteristic  of  class  B. 

4.  Recognition  objects.  As  a  result  of  solution  of  the  problem  formulated  in  p.  2 
with  max  e>0  we  obtain  the  table  of  codes  {xlj}. 

For  the  classification  of  a  certain  object  it  is  necessary  to  determine  n  values 
characterizing  its  parameters  and  to  find  for  each  gradation  the  code  corresponding  to  it. 
Let  us  obtain  n  numbers.  If  the  sum  of  these  numbers  proves  to  be  more  than  n/2,  then 
the  object  belongs  to  class  A;  if  is  less  than  n/2,  then  to  class  B.  The  object  is  not  identified 
when  there  is  a  coincidence  of  the  sum  of  the  codes  with  n/2. 

The  method  described  above  was  applied  in  the  computer  center  of  the  University 
of  Perm  to  compile  recognition  tables  for  dividing  beds  into  the  oil-bearing  and 
water-bearing  according  to  the  data  of  geophysical  measurements  in  the  Perm  region.  In 
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one  of  the  problems,  in  order  to  characterize  a  bed,  13  parameters  were  measured;  these 
included  the  time  of  opening  a  well,  thickness  of  bed,  diameter  of  well,  apparent  resistance 
measured  by  probes  of  different  lengths,  etc.  Each  parameter  had  from  2  to  6  gradations. 
In  this  problem  n=13,  and  the  total  number  of  gradations  N- 54.  For  teaching  purposes  we 
presented  on  30  oil-bearing  and  aquifers,  m,=m2=30.  Thus,  this  problem  was  reduced  to 
definition  of  55  unknowns  with  60  limits  of  type  (23),  (23’)  and  54  limits  of  type  (2.1).  In 
the  teaching  sequence  two  gradations  of  the  parameters  were  encountered  only  in  class  A 
and  3  gradations  only  in  class  B,  while  one  gradation  was  not  encountered  in  any  of  the 
objects.  The  remaining  gradations  (48  gradations)  were  encountered  in  both  classes.  As 
a  result  of  the  solution  a  recognition  table  was  obtained;  this  was  used  for  recognizing  beds 
with  no  rote  in  teaching. 

It  is  extremely  important  here  to  have  simplicity  of  recognition.  The  interpretation 
of  geophysical  measurements  in  a  well  can  be  done  with  the  help  of  the  obtained  tables 
directly  at  the  site  of  measurements  and  does  not  require  a  highly  skilled  person. 

5.  Observations  about  other  methods  of  optimization.  There  other  criteria  of 
optimality  possible.  For  example,  it  is  possible  to  maximize  the  total  distance  from  the 
dividing  hyperplane  to  all  points  of  the  teaching  sequence.  In  this  case  the  linear  form  with 
ml=m2=m  is 

2m  n  *• 

z  =  2  2  2  c<V  X<> 

r~*l  t—t  i"*l 


with  the  limits 


2  ^  ,  r—  1.2, ...,2m. 

,«>i 

0  <  xa  ^  1. 


Page  169 

In  the  solution  of  this  problem  it  is  possible  to  have  cases  where  some  objects  of  the 
teaching  sequence  prove  to  be  on  the  dividing  hyperplane,  and  the  teaching  sequence  will 
be  divided  incompletely  with  max  z>0. 
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If  necessary  it  is  possible  to  superimpose  on  codes  some  further  limits  with  the 


form 


x,j=fi(j.  ti,  t2, t.)  (5.1) 

where  f;  -  preset  linear  function  of  parameters  t„  t2, t,. 

The  value  of  the  code  Xg  will  be  determined  if  the  parameters  t1}  t2,  ...,  t,  are  found. 

The  problem  in  p.  2  will  preserve  linearity  with  substitution  of  relation  (5.1)  in  the 
limits  (2.1),  (23),  (23’),  but  if  s<N,  then  the  quantity  of  variables  sought  is  reduced,  and 
max  e  can  only  decrease;  also  narrowed  somewhat  will  be  the  class  of  sets  divided  by  one 
plane. 
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